Image generation apparatus, head-mounted display, content processing system, and image display method

ABSTRACT

The present disclosure provides a head-mounted display that includes an image generation device generating a display image to be viewed through an eyepiece disposed in front of a display panel, the image generation device including a source image reception control section that receives a source image, a distorted image generation section that generates data on pixels in a display image obtained by giving the source image a distortion corresponding to the eyepiece, a partially distorted image storage section that stores the data on the pixels in an order of data generation, and an image display control section that, whenever data on a predetermined number of pixels smaller than a total number of pixels in the display image is stored in the partially distorted image storage section, outputs the stored data to the display panel; the display panel that sequentially displays data outputted from the image generation device; and a content processing device that generates the source image and transmits the generated source image to the head-mounted display.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2019-185339 filed Oct. 8, 2019, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

The present disclosure relates to a head-mounted display for displayingan image in front of the eyes of a user wearing the head-mounteddisplay, an image generation device for processing a display image, acontent processing system, and an image display method.

Image display systems enabling a user to view a target space from adesired point of view are in widespread use. Developed, for example, arethe systems for displaying a panoramic image on a head-mounted displayand displaying an image in accordance with the gaze direction of a userwearing a head-mounted display. Using the head-mounted display makes itpossible to enhance the sense of immersion and improve the operabilityof a game or other application. In addition, a walk-through system isdeveloped. When a user wearing the head-mounted display physicallymoves, the walk-through system enables the user to virtually walk in aspace displayed as an image.

SUMMARY

The head-mounted display is featured in that it displays an image infront of the eyes of a viewer. Therefore, in order to providecomfortable viewing at a wide viewing angle, it is demanded that thehead-mounted display generate a display image in a different manner froman ordinary image displayed on a flat-panel display. In the case ofcontent created for viewing through the head-mounted display, such aspecial display image is generated during a process performed within thecontent.

Meanwhile, in a case where the head-mounted display is adapted to enablea user to view a large number of pieces of content created for displayon a flat-panel display, it is necessary to perform separate processingon an already-generated display image. However, such processing isdisadvantageous in terms, for example, of additional processing load,video delay caused by such processing, consumption of individualresources, responsiveness, and power consumption. Consequently, suchprocessing is not easily implementable.

In view of the above circumstances, the present disclosure has been madeto provide a technology for enabling a head-mounted display to displayvideo not ready for display on the head-mounted display with a smalldelay, at a low cost, and with a low power consumption.

According to an embodiment of the present disclosure, there is providedan image generation device that generates a display image to be viewedthrough an eyepiece disposed in front of a display panel. The imagegeneration device includes a source image reception control section, adistorted image generation section, a partially distorted image storagesection, and an image display control section. The source imagereception control section receives a source image. The distorted imagegeneration section generates data on pixels in the display image that isobtained by giving the source image a distortion corresponding to theeyepiece. The partially distorted image storage section stores the dataon the pixels in the order of data generation. Whenever data on apredetermined number of pixels smaller than the total number of pixelsin the display image is stored in the partially distorted image storagesection, the image display control section outputs the stored data tothe display panel.

In the image generation device, the distorted image generation sectionmay generate the data on the pixels in the display image by referencinga map indicating, on an image plane, a positional relationship betweenpixels in a distorted display image stored in the partially distortedimage storage section and pixels in the source image or by calculatingthe positional relationship.

The image generation device may further include a data transfer controlsection. Whenever data on the predetermined number of pixels is storedin the partially distorted image storage section, the data transfercontrol section exercises control so as to transmit the data.

The image generation device may further include a user posturecalculation section, a user posture value storage section, and a viewscreen projection determination section. The user posture calculationsection acquires information regarding the posture of the head of a userwearing a head-mounted display having the display panel. The userposture value storage section stores the acquired information regardingthe posture of the head of the user. The view screen projectiondetermination section sets a view screen that defines the plane of thedisplay image in accordance with the posture of the head of the user.The distorted image generation section may give the distortion to animage projected on the view screen.

The image generation device may further include a user controllerinstruction reception control section and a user controller input valuestorage section. The user controller instruction reception controlsection acquires a user instruction that determines whether the plane ofthe source image is to be fixed in a virtual space of a display targetor fixed to the display panel. The user controller input value storagesection stores the user instruction. In a mode for fixing the plane ofthe source image in the virtual space of the display target, thedistorted image generation section may give the distortion to the imageprojected on the view screen.

In the image generation device, the map may indicate the positionalrelationship at discrete positions having fewer pixels than the displayimage. Based on a positional relationship obtained by interpolating thepositional relationship indicated by the map, the distorted imagegeneration section may generate data on all pixels in the display image.

In the image generation device, the distorted image generation sectionmay generate the data on the pixels in the display image from the sourceimage by combining a transformation based on the map determined by thestructure of the eyepiece with a transformation based on a parameter notindicated by the map. At least either one of a pupillary distance of theuser and the distance between the display panel and the eyes of the usermay be used as the parameter not indicated by the map.

In the image generation device, the partially distorted image storagesection may include a plurality of storage areas each having a capacityfor storing data on the predetermined number of pixels. The distortedimage generation section may switch the storage location of the data onthe pixels in the display image between the plurality of storage areas.

In the image generation device, the plurality of storage areas includedin the partially distorted image storage section may each have acapacity that is an integer multiple of the capacity of a unit areadefined as a minimum unit of processing in the distorted imagegeneration section.

In the image generation device, the source image may be an undistortedimage generated for display on a flat-panel display.

According to another mode of the present disclosure, there is provided ahead-mounted display including the above-described image generationdevice that generates a display image to be viewed through an eyepiecedisposed in front of a display panel, the image generation deviceincluding a source image reception control section that receives asource image, a distorted image generation section that generates dataon pixels in a display image obtained by giving the source image adistortion corresponding to the eyepiece, a partially distorted imagestorage section that stores the data on the pixels in an order of datageneration, and an image display control section that, whenever data ona predetermined number of pixels smaller than a total number of pixelsin the display image is stored in the partially distorted image storagesection, outputs the stored data to the display panel. The display panelsequentially displays data that is transmitted from the image generationdevice.

According to yet another mode of the present disclosure, there isprovided a content processing system including the above-describedhead-mounted display that includes an image generation device generatinga display image to be viewed through an eyepiece disposed in front of adisplay panel, the image generation device including a source imagereception control section that receives a source image, a distortedimage generation section that generates data on pixels in a displayimage obtained by giving the source image a distortion corresponding tothe eyepiece, a partially distorted image storage section that storesthe data on the pixels in an order of data generation, and an imagedisplay control section that, whenever data on a predetermined number ofpixels smaller than a total number of pixels in the display image isstored in the partially distorted image storage section, outputs thestored data to the display panel; the display panel that sequentiallydisplays data outputted from the image generation device; and a contentprocessing device. The content processing device generates the sourceimage and transmits the generated source image to the head-mounteddisplay.

In the content processing system, when a source image transmitted fromthe content processing device is displayable as is, the image displaycontrol section may output data on the source image to the displaypanel.

In the content processing system, at a time point when data on pixels inrows of a frame of the source image transmitted from the contentprocessing device is acquired as necessary for determining one row ofpixel values of the display image, the distorted image generationsection starts a process of generating data on the one row of pixels.

According to still another mode of the present disclosure, there isprovided an image display method used in an image generation device forgenerating a display image that is to be viewed through an eyepiecedisposed in front of a display panel. The image display method includes:receiving a source image; generating data on pixels in the display imagethat is obtained by giving the source image a distortion correspondingto the eyepiece, and sequentially storing the generated data in amemory; and, whenever data on a predetermined number of pixels smallerthan the total number of pixels in the display image is stored in thememory, outputting the stored data to the display panel.

Any combinations of the above-mentioned constituent elements and anyconversions of expressions of the present disclosure, for example,between methods, devices, systems, computer programs, data structures,and recording media are also valid modes of the present disclosure.

According to the above-mentioned modes of the present disclosure, videonot ready for display on a head-mounted display can be displayed on thehead-mounted display with a small delay, at a low cost, and with a lowpower consumption.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an example external view of a head-mounted display accordingto an embodiment of the present disclosure;

FIG. 2 is a diagram illustrating an example configuration of a contentprocessing system according to the embodiment;

FIG. 3 is a diagram illustrating a process necessary for causing thehead-mounted display to display an image that is to be displayed on aflat-panel display;

FIG. 4 is a diagram illustrating a circuit configuration of thehead-mounted display according to the embodiment;

FIG. 5 is a diagram illustrating two modes that can be implemented inthe embodiment to display a source image;

FIGS. 6A-6B depict a set of conceptual diagrams illustrating imagechanges necessary for displaying a source image on the head-mounteddisplay in the embodiment;

FIG. 7 is a diagram illustrating a configuration of functional blocks ofan image processing device built in the head-mounted display accordingto the embodiment;

FIG. 8 is a diagram illustrating a process that is performed by adistorted image generation section in the embodiment in order togenerate a distorted display image from an undistorted source image;

FIG. 9 is a diagram illustrating processing steps performed by thehead-mounted display according to the embodiment;

FIG. 10 is a diagram illustrating an example structure of a partiallydistorted image storage section in the embodiment; and

FIGS. 11A-11B illustrate the significance of the embodiment in terms ofthe time required for processing an undistorted image and displaying theprocessed image.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 illustrates an example external view of a head-mounted display100 according to an embodiment of the present disclosure. In theillustrated example, the head-mounted display 100 includes an outputmechanism section 102 and a wearing mechanism section 104. The wearingmechanism section 104 includes a wearing band 106. When worn by a user,the wearing band 106 surrounds the head of the user so as to secure thehead-mounted display 100. The output mechanism section 102 includes ahousing 108. The housing 108 is shaped so as to cover the left and righteyes of the user when the user wears the head-mounted display 100. Thehousing 108 includes a display panel that faces the eyes of the userwhen the user wears the head-mounted display 100.

The housing 108 additionally includes an eyepiece. When the user wearsthe head-mounted display 100, the eyepiece is placed between the displaypanel and the eyes of the user, and makes an enlarged image visible tothe eyes of the user. Further, the head-mounted display 100 mayadditionally include speakers or earphones that are placed at positionscorresponding to those of the ears of the user. Furthermore, thehead-mounted display 100 has a built-in motion sensor that detectstranslational motions and rotational motions of the head of the userwearing the head-mounted display 100, and detects the position andposture of the user's head at each time point.

In the present example, the head-mounted display 100 includes stereocameras 110, a monocular camera 111 with a wide viewing angle, and fourother cameras 112 with a wide viewing angle, and shoots a video of areal space located in a direction corresponding to the orientation ofthe face of the user. The stereo cameras 110 are disposed on the frontsurface of the housing 108. The monocular camera 111 is disposed at thecenter of housing 108. The cameras 112 are disposed in four corners ofthe housing 108, namely, the upper left, upper right, lower left, andlower right corners of the housing 108. When an image captured by thestereo cameras 110 is instantly displayed, what is called videosee-through is achieved so that the user is able to directly view a realspace located in a direction in which the user is facing. Further,augmented reality is achieved when an image of a real object depicted inthe captured image is displayed in superimposition upon an image of avirtual object reacting with the real object. Furthermore, when at leastone of images captured by the above-mentioned seven cameras is analyzedby using a technology such as SLAM (simultaneous localization andmapping), it is possible to acquire information regarding the positionand posture of the head-mounted display 100 relative to a surroundingspace, and thus information regarding the position and posture of theuser's head. It is also possible, for example, to achieve objectrecognition and make an object depth measurement.

FIG. 2 illustrates an example configuration of a content processingsystem according to the present embodiment. The head-mounted display 100is connected to a content processing device 200 through wirelesscommunication or through an interface 300 connecting to a universalserial bus (USB) Type C or other peripheral. The content processingdevice 200 is connected to a flat-panel display 302. The contentprocessing device 200 may be further connected to a server through anetwork. In such a case, the server may supply an online application,such as a game in which a plurality of users can participate through thenetwork, to the content processing device 200.

The content processing device 200 basically processes a program ofcontent to generate a display image, and transmits the generated displayimage to the head-mounted display 100 and the flat-panel display 302. Ina certain mode, the content processing device 200 determines theposition of a point of view and the direction of gaze in accordance withthe position and posture of the head of the user wearing thehead-mounted display 100, and generates a display image of the contentat a predetermined rate so as to provide an appropriate field of view.

The head-mounted display 100 receives data on the display image anddisplays the received data as an image of the content. In such aninstance, the purpose of displaying an image is not particularlylimited. For example, the content processing device 200 may generate adisplay image depicting a virtual world serving as the stage of anelectronic game while allowing the electronic game to progress, or maydisplay a still image or a video image for purposes of appreciation orinformation provision no matter whether the displayed image depicts avirtual world or a real world.

The distance between the content processing device 200 and thehead-mounted display 100 and the method of communication provided by theinterface 300 are not limited. For example, the content processingdevice 200 may be a personally-owned gaming device, a server of acompany providing cloud game and various distribution services, or ahome server transmitting data to a terminal. Therefore, thecommunication between the content processing device 200 and thehead-mounted display 100 may be established not only by using the methoddescribed in the above example, but also through a network or an accesspoint such as the Internet or other public network, a local area network(LAN), a mobile phone carrier network, a Wi-Fi spot in town, or a homeWi-Fi access point.

FIG. 3 is a diagram illustrating a process necessary for causing thehead-mounted display 100 to display an image that is to be displayed onthe flat-panel display 302. The example of FIG. 3 is for displaying avirtual space where an object, such as a table, exists. In thisinstance, first of all, an image 16 corresponding to the field of viewof the user is drawn. A common computer graphics technology can beapplied to such drawing. The image 16 represents the image to bevisually recognized by the user, and is a common undistorted image.

When stereoscopic viewing is to be provided, a stereo image including aleft-eye image 18 a and a right-eye image 18 b is generated from theimage 16. The left-eye image 18 a and the right-eye image 18 b areimages obtained by shifting the horizontal position of an object by aparallax that is determined by the interval between the left and righteyes and the distance to the object. Next, a final display image 22 isgenerated by subjecting the left-eye image 18 a and the right-eye image18 b to inverse correction in accordance with a distortion caused by theeyepiece of the head-mounted display 100.

The above-mentioned inverse correction is a process of pre-distorting animage in a direction opposite to the distortion caused by a lens so thatthe original image 16 is visually recognizable when viewed through theeyepiece. For example, when the employed lens is such that the foursides of an image look as if they are concaved like a bobbin, the imageis curved like a barrel beforehand. An image given a distortioncorresponding to the employed lens is hereinafter referred to as a“distorted image.” For example, the head-mounted display 100 acquires,at a predetermined frame rate, the display image 22 generated by thecontent processing device 200, and displays the acquired display image22 as is on the display panel.

When the displayed image is viewed through a lens, the left eye of theuser visually recognizes the left-eye image 18 a, and the right eye ofthe user visually recognizes the right-eye image 18 b. As a result, avideo image of one frame of the image 16 is stereoscopically recognized.As regards content created for display on the head-mounted display 100,a process of generating a distorted image as the display image 22 asdescribed above is commonly performed as a part of a content program.

Meanwhile, as regards content created for display on a flat-paneldisplay or a screen, such as a conventional electronic game, a recordedvideo, a television program, or a movie, an undistorted image, such asthe image 16, is generated as a display image at a predetermined framerate. The same holds true for a system screen that is called by suchcontent. If such an image is displayed as is on the head-mounteddisplay, an image distorted by the eyepiece is visually recognized.

In view of the above circumstances, the present embodiment uses anintegrated circuit that generates a distorted image suitable for displayon the head-mounted display 100 from an undistorted image generated by aprocess within content. More specifically, the integrated circuit foroutputting the display image 22 by using the image 16 depicted in FIG. 3as input data is employed so that a large number of pieces ofconventional content created for display on the head-mounted display 100are easily viewable through the head-mounted display 100. This makes iteasy to fuse or superimpose an image with a content image created fordisplay on the head-mounted display 100.

When the above-described integrated circuit is incorporated in thehead-mounted display 100, the content processing device 200 simply hasto generate and output an undistorted image by processing conventionalcontent in a usual manner no matter whether the head-mounted display 100or the flat-panel display is used as a display destination. In somecases, an alternative is to incorporate the integrated circuit in thecontent processing device 200 and generate and output a distorted imagewhen the display destination is the head-mounted display 100. The formercase is hereinafter described as an example.

FIG. 4 illustrates a circuit configuration of the head-mounted display100 according to the present embodiment. However, FIG. 4 depicts onlythe elements according to the present embodiment, and does not depictthe other elements. The head-mounted display 100 includes aninput/output interface 30, a main memory 34, an image processingintegrated circuit 36, a motion sensor 48, an image sensor 50, and adisplay panel 46.

The input/output interface 30 establishes wired or wirelesscommunication with the content processing device 200 in order totransmit and receive data. In the present embodiment, the input/outputinterface 30 mainly receives image data from the content processingdevice 200 at a predetermined rate. An image transmitted as data fromthe content processing device 200 may be referred to as a “sourceimage.” The input/output interface 30 may further receive audio datafrom the content processing device 200, and transmit a value measured bythe motion sensor 48 and an image captured by the image sensor 50 to thecontent processing device 200.

The main memory 34 is a main storage section that is implemented, forexample, by a DRAM (dynamic random-access memory) in order to store, forinstance, data, parameters, and operation signals to be processed by acentral processing unit (CPU) 32. Data on the source image transmittedfrom the content processing device 200 is temporarily stored in the mainmemory 34.

The image processing integrated circuit 36 includes the CPU 32, agraphics processing unit (GPU) 38, a buffer memory 42, a displaycontroller 44, and a handshake controller 40. The CPU 32 controls theother circuits and the sensors. The GPU 38 draws a display image. Thebuffer memory 42 temporarily stores data on a drawn image. The displaycontroller 44 transmits the display image to the display panel 46. Thehandshake controller 40 controls the transmission timing of the displayimage. The CPU 32 is a main processor that processes and outputscommands, data, and signals, such as image signals and sensor signals,in order to control the other circuits and the sensors.

The GPU 38 corrects or otherwise processes the source image, which istransmitted from the content processing device 200 and stored in themain memory 34, in order to draw a final image to be displayed. However,the data on the source image need not be temporarily stored in the mainmemory 34. An alternative is to avoid a delay of processing and anincrease in the memory usage amount by allowing the GPU 38 to startprocessing without causing the main memory 34 to store the data on thesource image. Further, a drawing process may be performed either by theCPU 32 or by the CPU 32 cooperating with the GPU 38. However, thefollowing description is given as an example on the assumption that thedrawing process is performed by the GPU 38.

The GPU 38 basically performs the drawing process on each of unit areas,which are formed by dividing an image plane, in order to referencetexture data and perform TBDR (tile-based deferred rendering). Here, itis assumed that the unit areas are rectangular areas, for example, of32×32 pixels or 64×64 pixels. The GPU 38 starts the drawing process uponeach acquisition of data on the unit areas to be processed.

In a case where the source image is an undistorted image representativeof each frame of video, the GPU 38 generates the left-eye image and theright-eye image as mentioned earlier, and applies inverse correction oflens distortion to each of the left- and right-eye images in order togenerate an image formed by connecting the left- and right-eye images toeach other. The GPU 38 may separately acquire a setting for the distanceto an object to be displayed and generate the left- and right-eye imagesexactly in accordance with a parallax corresponding to the acquiredsetting for the distance, or may generate the left- and right-eye imagesby performing a simple process, for example, of applying a uniformparallax to all images. In any case, the GPU 38 generates the left- andright-eye images in such a manner that they look natural when viewedthrough an eyepiece having an optical axis of each of them.

The GPU 38 determines pixel values in raster order in accordance withthe order in which the elements of the display panel 46 are driven. Morespecifically, from the top to the bottom of the image plane, the GPU 38repeatedly performs a process of sequentially determining the pixelvalues in the rightward direction beginning with the upper left pixel inthe image plane. For this purpose, a map indicating the positionalrelationship between pixels in an undistorted source image andcorresponding pixels in a distorted display image is prepared. Then, foreach pixel in the display image, the GPU 38 reads a plurality of valuesof neighboring pixels near a corresponding position in the source image,and performs a filtering process to determine the pixel value. For thefiltering process of determining one pixel value by using neighboringpixels, various computation methods are proposed. Any one of suchcomputation methods may be used.

However, a transformation formula for determining the positionalrelationship between pixels in a distorted image and pixels in thesource image may be set instead of the above-mentioned map. Further, afactor for determining the pixel values of the display image is notlimited to pixel displacement caused by the presence of distortion. Forexample, in a mode in which a three-dimensional virtual space where thesource image is fixed is viewed through a head-mounted display, theposition and posture of the source image relative to a display screenvaries with the position and posture of the head-mounted display, thatis, the position and posture of the user. Therefore, it is necessary toperform a process of projecting the plane of the source image, which isfixed in the three-dimensional space, to the view screen of the displayscreen, which is determined by the position and posture of thehead-mounted display.

Accordingly, the pixel values are determined by combining the followingparameters as appropriate with information regarding distortiondetermined based on the structure of the eyepiece.

-   -   1. The posture and orientation of the user based on an output        value of the motion sensor 48 and the result of SLAM        calculation.    -   2. The pupillary distance specific to the user (the distance        between the left and right eyes of the user).    -   3. A parameter determined as a result of adjusting the wearing        mechanism section 104 (wearing band 106) of the head-mounted        display 100 in accordance with the relationship between the head        and eyes of the user.

The pupillary distance mentioned under “2” above is acquired in a mannerdescribed below. In a case where the head-mounted display 100 includesgaze tracking stereo cameras, the gaze tracking stereo cameras capturean image of the pupils of the eyes of the user wearing the head-mounteddisplay 100. As an alternative, the user may orient, for example, thestereo cameras 110 disposed on the front surface of the head-mounteddisplay 100 toward the face of the user and capture an image of the facewith open eyes. As another alternative, an undepicted camera outside thecontent processing system may be oriented toward the user to capture animage of the face with open eyes. The pupillary distance is thenautomatically measured and recorded by processing the image captured inthe above-described manner through the use of image recognition softwarefor the pupils of the eyes, which runs on the content processing system.

In a case where the camera-to-camera distance between the gaze trackingstereo cameras or between the stereo cameras 110 is used, triangulationis performed. As an alternative, the content processing system maydisplay the captured image on the flat-panel display 302, allow the userto specify the positions of the pupils of the left and right eyes, andlet the content processing device 200 calculate and record the pupillarydistance between the left and right eyes in accordance with thespecified positions of the pupils. Alternatively, the user may directlyregister the user's pupillary distance. The pupillary distance acquiredin the above-described manner is then reflected on the distance betweenthe left- and right-eye images within the display image 22 depicted inFIG. 3 .

As regards “3” above, mechanical adjustment results concerning thewearing mechanism section 104 and the wearing band 106 are acquired byundepicted measuring instruments built in the head-mounted display 100,such as a rotary encoder and a variable resistor. The content processingsystem calculates the distance and angle between the eyepiece and theeyes of the user in accordance with the adjustment results. Theparameter acquired in the above-described manner is reflected on themagnifications and positions of the images within the display image 22depicted in FIG. 3 .

The parameters described under “1” to “3” above are specific to the userwearing the head-mounted display 100 or variable with the position andposture of the user. Therefore, the parameters are not easily reflectedon the map beforehand. Consequently, the GPU 38 may determine the finalpixel values by combining a transformation based on the map with atransformation based on at least one of the parameters described under“1” to “3.”

The GPU 38 sequentially stores the pixel values in the buffer memory 42in the order of pixel value determination. The GPU 38 monitors thenumber of processed pixels. The buffer memory 42 monitors the number ofpixels whose values are stored. The display controller 44 monitors thenumber of outputted pixels. Under the control of a data transfer controlsection in the CPU 32, the handshake controller 40 constantly monitors aposition in the buffer memory 42 in which the GPU 38 writes data and aposition in the buffer memory 42 from which the display controller 44reads data, and prevents a data deficiency, that is, a buffer underrun,and a data overflow, that is, a buffer overrun. Upon detecting a statewhere a buffer overrun may occur, the handshake controller 40 issues aninstruction for data output suppression to the GPU 38.

Upon detecting a state where a buffer underrun may occur, the handshakecontroller 40 issues an instruction for data output acceleration to theGPU 38. If a buffer underrun or a buffer overrun occurs, it is reportedto the data transfer control section operating in the CPU 32. The datatransfer control section notifies the user of the occurrence of anabnormality and performs a transfer resumption process. Accordingly,whenever data on a predetermined number of pixels (hereinafter referredto as the transmission unit data) smaller than the total number ofpixels in the display image is stored in the buffer memory 42 under thecontrol of the GPU 38, the stored data is transmitted from the displaycontroller 44 to the display panel 46.

The transmission unit data is, for example, the data on one row of thedisplay image or the data on a plurality of rows that are obtained byequally dividing all the rows of the display image. In a case where thetransmission unit data is the data on the plurality of rows, the size ofthe data is an integer multiple of the above-mentioned unit area definedas a minimum unit of processing for drawing by the GPU 38. As anexample, data on a number of rows obtained by dividing the display imageby 16 is assumed to be the transmission unit data.

As far as the data is transmitted in units smaller than the overallframe of the display image as described above, the actual time pointwhen the handshake controller 40 establishes the communication betweenthe GPU 38 and the display controller 44 is not particularly limited. Ina case where the transmission unit data is the data on the plurality ofrows, the GPU 38 does not have to determine the pixel values in rasterorder. Instead, the GPU 38 may perform processing by using a tile formatobtained by bundling the plurality of rows. Within the transmission unitdata, the order of pixel value determination may be different from theraster order.

As the above-described configuration is adopted, the size of the buffermemory 42 can be made significantly smaller than the size of one frameof the display image. For example, two storage areas for storing thetransmission unit data, one for reading and the other for writing, maybe prepared, or two or more ring buffers may be prepared as a storagearea for storing the transmission unit data. In such a case, the buffermemory 42 can be incorporated in the image processing integrated circuit36 as a SRAM (static random-access memory). This makes it possible toprovide rapid access and reduce power consumption as compared with amode in which a frame buffer is disposed in the main memory 34 such as aDRAM. The buffer memory 42 may be configured integrally with the mainmemory 34.

The display controller 44 displays an image by sequentially convertingthe transmission unit data read from the buffer memory 42 to electricalsignals and driving the pixels of the display panel 46 at an appropriatetime point. In a case where an initially distorted source image is used,the display controller 44 should display the source image on an as-isbasis by similarly processing the source image stored in the main memory34 to drive the display panel 46. However, in order to avoid a delay ofprocessing and an increase in the memory usage amount, the source imagemay be processed without being stored in the main memory 34.

Information indicating whether the source image is ready for display onan as-is basis is transmitted from the content processing device 200 asadditional data on the source image, and recognized by the CPU 32 thatchanges a subsequent operation accordingly. The display panel 46 has acommon display mechanism such as a liquid-crystal display or an organicelectroluminescence (EL) display, and displays an image in front of theeyes of the user wearing the head-mounted display 100. When the userviews the image through the eyepiece, the user visually recognizes anundistorted image.

The motion sensor 48 detects posture information such as the rotationangle and inclination of the head-mounted display 100. The motion sensor48 is implemented by combining, for example, a gyro sensor, anacceleration sensor, and an angular acceleration sensor as appropriate.The image sensor 50 corresponds to the stereo cameras 110 depicted inFIG. 1 , and captures an image of the real world in the field of viewcorresponding to the position and orientation of the face of the user.The image sensor 50 is not limited to the stereo cameras 110, and may beone of the monocular camera 111 and the four other cameras 112 or acombination of them. The head-mounted display 100 may additionallyinclude, for example, an audio circuit for allowing the user to listento a sound and a peripheral interface circuit for connecting to aperipheral.

FIG. 5 is a diagram illustrating two modes that can be implemented inthe present embodiment to display the source image. Two modes, namely, a“head space mode” and a “world space mode,” can be implemented in thepresent embodiment. In the former mode, the plane of the source image isfixed with respect to the face of the user, that is, the display panel46 of the head-mounted display 100. In the latter mode, the plane of thesource image is fixed with respect to the virtual space of the displaytarget as mentioned earlier. Meanwhile, as depicted in the perspectiveview 70 of FIG. 5 , a yaw angle, a pitch angle, and a roll angle aredefined relative to three axes of the head of the user. These parametersare acquired based on the output value of the motion sensor 48 and theresult of SLAM calculation.

As depicted on the right side of FIG. 5 , the source image in the headspace mode coordinates with the orientation of the face of the user.Therefore, in the head space mode, the displacement of pixels toward thedisplay image can be determined without regard to changes in the yawangle, the pitch angle, and the roll angle. In the world space mode,however, the relative angle between the view screen for the displayimage and the plane of the source image varies with the changes in theyaw angle, the pitch angle, and the roll angle. Consequently, thedisplacement of pixels is dynamically determined based on suchparameters.

FIGS. 6A-6B conceptually illustrate image changes necessary fordisplaying the source image on the head-mounted display. FIG. 6A relatesto the head space mode, and FIG. 6B relates to the world space mode. Inboth of these two modes, a stereo image 74 on which a parallax isreflected is obtained from a source image 72, as is the case with theprocess described with reference to FIG. 3 . In the head space modedepicted in FIG. 6A, the posture of the view screen relative to theplane of the source image remains unchanged. Therefore, a display image76 is obtained by distorting the left and right images within the stereoimage 74, as is the case with the process described with reference toFIG. 3 .

In the world space mode depicted in FIG. 6B, the posture of the viewscreen relative to the source image varies with the posture of the headof the user. Therefore, a display image 78 is obtained by setting theview screen in accordance with the yaw angle, the pitch angle, and theroll angle, projecting the source image, and giving the projected sourceimage to a distortion corresponding to the eyepiece. The broken linedepicted within the display image 78 represents the boundary of thefield of view, and is not actually displayed. When the above-describedimage is displayed, it looks as if a source image screen is floating inthe darkness.

One of two depicted modes may be selected directly by the user orindirectly selected based on a change in another parameter. For example,one of the two modes may be selected based on the size of the sourceimage displayed on the screen of the head-mounted display 100. In such acase, the world space mode may be selected if the size of the displayedsource image is larger than a threshold value, and the head space modemay be selected if the size of the displayed source image is equal to orsmaller than the threshold value. These two modes and switching betweenthem are disclosed in International Publication WO 2017/051564.

FIG. 7 illustrates a configuration of functional blocks of an imageprocessing device 128 built in the head-mounted display 100. Thedepicted functional blocks can be implemented by hardware such as theimage processing integrated circuit 36 depicted in FIG. 4 or implementedby software including programs loaded into the main memory 34, forexample, from a recording medium in order to exercise various functionssuch as a data input function, a data retention function, an imageprocessing function, and a communication function. Therefore, it will beunderstood by those skilled in the art that the functional blocks may bevariously implemented by hardware alone, by software alone, or by acombination of hardware and software and are not to be limited to any ofthem.

A user controller instruction reception control section 132 included inthe image generation device 128 is implemented by the CPU 32 and theinput/output interface 30, and used to acquire information regarding auser operation for selecting either the head space mode or the worldspace mode. For example, the user inputs information for selectingeither the head space mode or the world space mode, and then the usercontroller instruction reception control section 132 acquires theuser-inputted information. A user controller input value storage section136 eventually stores the acquired user-inputted information.

A user posture calculation section 134 is implemented, for example, bythe CPU 32, the motion sensor 48, and the image sensor 50, and used toacquire information regarding the posture of the head of the userwearing the head-mounted display 100, that is, the above-mentioned yawangle, pitch angle, and roll angle. As described earlier, theseparameters can also be acquired by analyzing a captured image instead ofthe value measured by the motion sensor 48. The analysis may be made bythe content processing device 200. A user posture value storage section138 stores information regarding the posture of the head of the user. Aview screen projection determination section 152 is implemented by theCPU 32 and the GPU 38, and used in the world space mode to set the viewscreen that defines the plane of the display image in accordance withthe posture of the head of the user.

A view screen position information storage section 150 storesinformation regarding the position and posture in a virtual space of theview screen, which is set in the above-described manner. A source imagereception control section 130 is implemented by the CPU 32 and theinput/output interface 30, and used to receive a source image createdfor display on a flat-panel display. A displacement vector map storagesection 140 stores a displacement vector map that is depicted in animage plane to indicate the positional relationship (displacementvector) between pixels in the display image given a distortioncorresponding to the eyepiece and the corresponding pixels in the sourceimage. A distorted image generation section 144 is implemented by theCPU 32 and the GPU 38, and used to generate data on the pixels in thedisplay image that is obtained by giving the source image at least thedistortion corresponding to the eyepiece.

In the world space mode, the distorted image generation section 144projects a stereo image on the view screen as depicted in FIG. 6B anddistorts the projected image in accordance with the informationregarding the position and posture of the view screen, which is storedin the view screen position information storage section 150. Further,irrespective of the selected mode, the distorted image generationsection 144 controls the interval between the left and right images andtheir magnifications during the generation of the stereo image inaccordance with the pupillary distance and the distance and anglebetween the eyepiece and the eyes of the user, which are mentionedabove.

A partially distorted image storage section 142 is implemented by thebuffer memory 42, and used to store corrected pixel data in the order ofpixel data generation. A data transfer control section 146 isimplemented by the CPU, the handshake controller, and the buffer memory,and used to exercise control so as to transmit data on a predeterminednumber of pixels smaller than the total number of pixels in the displayimage each time the data is stored in the partially distorted imagestorage section 142. An image display control section 148 is implementedby the display controller 44 and the display panel 46, and used todisplay an image in accordance with the transmitted pixel data.

FIG. 8 is a diagram illustrating a process that is performed by thedistorted image generation section 144 in order to generate a distorteddisplay image from an undistorted source image. As described thus far, adistorted display image 62 to be displayed on the head-mounted display100 is an image that is visually recognized as an undistorted image 60when viewed through the eyepiece. If distorted images within the displayimage 62 are to be transformed to the undistorted image 60, processingperformed for such a transformation is equivalent to a process ofcorrecting a camera lens distortion in a common captured image. That is,the displacement vector (Δx, Δy) of a pixel at position coordinates (x,y) concerning the above transformation can be calculated from thefollowing general equation.[Math. 1]Δx=(k ₁ r ² +k ₂ r ⁴ +k ₃ r ⁶+ . . . )(x−c _(x))Δy=(k ₁ r ² +k ₂ r ⁴ +k ₃ r ⁶+ . . . )(y−c _(y))  (Equation 1)

In the above equation, r represents the distance between the opticalaxis of the eyepiece and a target pixel, and (Cx, Cy) represents theposition of the optical axis of the eyepiece. Further, k₁, k₂, k₃, andso on are lens distortion coefficients and dependent on the design ofthe eyepiece. The degree of correction is not particularly limited.However, it should be noted that the equation used for correction is notlimited to the above. In order to consider the horizontal displacementbetween an image within the source image and the images in the left- andright-eye areas of the display image 62 in addition to theabove-described image distortion, the displacement Δd is added to Δx inEquation 1. The value Δd is a constant based, for example, on theinterval between the optical axes of lenses of the eyepiece.

Calculating the displacement vector (Δx, Δy) in the above-describedmanner clarifies the position coordinates (x+Δx, y+Δy) of pixel B in theimage 60 (source image), which corresponds to pixel A at the positioncoordinates (x, y) of the display image 62. The displacement vector (Δx,Δy) can be calculated beforehand as a function of the positioncoordinates (x, y) of the pixel in the display image. Therefore, thedisplacement vector map storage section 140 is disposed, for example, inthe main memory 34 to store the displacement vector map, which indicatesthe displacement vector (Δx, Δy) with respect to the image plane of thedisplay image.

The displacement vector map may indicate the displacement vector withrespect to all pixels in the display image or indicate the displacementvector only at discrete positions having fewer pixels than the displayimage. In the latter case, the distorted image generation section 144acquires the displacement vector for each pixel in the display image byinterpolating the displacement vector indicated by the displacementvector map.

Subsequently, the distorted image generation section 144 determinestarget pixels in the plane of the display image in raster order,identifies the corresponding positions in the source image in accordancewith the displacement vector map, reads a plurality of values ofneighboring pixels near pixels near the identified correspondingpositions, and performs a filtering process on the read pixel values todetermine the pixel values of the target pixels. For the filteringprocess of determining one pixel value by using neighboring pixels,various computation methods are proposed. Any one of such computationmethods may be used. As a result, the pixel values can be determined onan individual pixel basis. Therefore, data transmission can be performedin units smaller than the whole image. However, as mentioned earlier,the pixel values may be determined by combining, as appropriate, theposture of the head-mounted display 100 and various parameters specificto the user.

Further, in some cases where a correction is to be made for lensdistortion by using Equation 1, the eyepiece distortion coefficients mayfrom one color to another. It should be noted that not only a commonconvex lens but also a Fresnel lens may be used as the eyepiece mountedon the head-mounted display 100. Although the Fresnel lens can bethinned, it is susceptible not only to resolution degradation but alsoto image distortion that concentrically increases toward the peripheryof the field of view. Thus, nonlinear brightness changes may occur.Characteristics of such concentric brightness changes may vary withcolor, namely, red, green, or blue (refer, for example, to “Distortion,”Edmund Optics Technical Data; [online] URL:https://www.edmundoptics.jp/resources/application-notes/imaging/distortion/reference).Furthermore, while the light-emitting elements of the pixels of thedisplay panel have various color arrays, it is necessary that positionson the image plane agree with positions on the display panel on anindividual subpixel basis. Accordingly, the displacement vector map mayinclude components for making necessary corrections on an individualcolor basis.

Moreover, when a liquid-crystal panel is used as the display panel 46,the reaction speed is low although a high resolution can be achieved.Meanwhile, when an organic EL panel is used, a high resolution may noteasily be achieved although the reaction speed is high, and what iscalled “Black Smearing” is likely to occur. Black Smearing is aphenomenon in which color bleeding occurs in a black region and itsperiphery. In addition to the above-mentioned correction for lensdistortion, the distorted image generation section 144 may makecorrections so as to avoid the above-described various adverse effectsproduced by the eyepiece and the display panel. In such an instance, thedistorted image generation section 144 internally retains thecharacteristics of the display panel 46 together with thecharacteristics of the eyepiece. When, for example, a liquid-crystalpanel is used, the distorted image generation section 144 resets liquidcrystal by inserting a black image between frames, and thus increasesthe reaction speed. Further, when an organic EL panel is used, thedistorted image generation section 144 offsets a brightness value and agamma value for gamma correction in such a manner that color bleedingcaused by Black Smearing is made inconspicuous.

FIG. 9 is a diagram illustrating processing steps performed by thehead-mounted display 100. First of all, the content processing device200 transmits an undistorted image 60 at a predetermined rate. Thesource image reception control section 130 in the head-mounted display100 then acquires data on the image 60.

Subsequently, the distorted image generation section 144 generates adisplay image 62 that is given a distortion in the manner described withreference to FIGS. 6A-6B. In this instance, the distorted imagegeneration section 144 may start generating the display image 62 withoutwaiting until one complete frame of the image 60 is acquired. At a timepoint when data on a required number of rows is acquired for determiningthe pixel values of a row of the display image 62, the drawing of therow of the display image 62 can be started to further reduce latencyassociated with display.

In any case, the partially distorted image storage section 142 (buffermemory 42) sequentially stores data on pixels beginning with theuppermost row of the display image 62. At a time point when thetransmission unit data is stored in the partially distorted imagestorage section 142 under the timing control of the data transfercontrol section 146, the image display control section 148 reads anddisplays the transmission unit data. When, for example, the transmissionunit data 64 on the display image 62 is stored at a certain time point,a corresponding electrical signal drives the associated row of thedisplay panel 46. Subsequently, the same process is repeatedly performedin a direction toward the bottom of the image until the display image 62is entirely displayed.

FIG. 10 illustrates an example structure of the partially distortedimage storage section 142. In the illustrated example, the partiallydistorted image storage section 142 includes a first storage area 66 anda second storage area 68. The first and second storage areas 66 and 68are both sized to match the size of the transmission unit data. The datais written into one storage area. When the size of the transmission unitdata is reached, the data in the one storage area begins to be readwhile the next data is written into the other storage area. This processis repeated so as to write and read the data while interchanging theroles of the two storage areas.

As mentioned earlier, it is assumed that the unit of transmission to thedisplay panel 46 in a certain mode is an integer multiple of a unit areadefined as the minimum unit of texture data referencing during thedrawing process of the GPU 38 forming the distorted image generationsection 144 and processing for tile-based deferred rendering. In thiscase, the first storage area 66 and second storage area 68 of thepartially distorted image storage section 142 each have a capacity thatis an integer multiple of the unit area.

Three or more areas for individual transmission unit data may bedisposed in the partially distorted image storage section 142 andcyclically used. Using three or more storage areas makes it possible toprevent a failure where a read-out is delayed due to a subtle speeddifference between a data write-in and a data read-out so that data notread out yet is soon overwritten by new data.

FIGS. 11A-11B illustrate the significance of the present embodiment interms of the time required for processing an undistorted image anddisplaying the processed image. In FIGS. 11A-11B, the horizontaldirection represents elapsed time, the solid-line arrows represent thetime of display image drawing by the distorted image generation section144, and the broken-line arrows represent the time of output to thedisplay panel 46. As regards parenthesized descriptions affixed to“DRAWING” or “OUTPUT,” (m) indicates a process performed on one framehaving the frame number m, and (m/n) indicates the nth process performedon transmission unit data having the frame number m. FIG. 11A indicates,as a comparative example, a mode in which one frame of display image isstored in the main memory 34 before being outputted to the displaypanel.

More specifically, during a period between time t0 and time t1, thefirst frame is drawn and its data is stored in the main memory 34. Attime t1, the second frame begins to be drawn, and the data on the firstframe is sequentially read from the main memory 34 and outputted to thedisplay panel 46. Such processing is completed at time t2, and then thethird frame is drawn and the second frame is outputted. Subsequently,the individual frames are drawn and outputted in the same cycle. In suchan instance, the time required between the start of drawing of one frameof display image and the completion of its output is equal to the outputcycle of two frames.

According to the present embodiment depicted in FIG. 6B, at a time pointwhen the drawing of the first transmission unit data on the first frameis completed, it is read from the partially distorted image storagesection 142 and outputted to the display panel 46. During such a period,the second transmission unit data is drawn. Therefore, subsequently tothe first transmission unit data, the second transmission unit data canbe outputted to the display panel 46. When such an operation isrepeated, at time t1 at which the drawing of the last (nth) transmissionunit data is completed, the output of the second last (n−1th)transmission unit data is already terminated. The subsequent frames aresimilarly outputted to the display panel 46 in parallel with the drawingprocess.

Consequently, the time required between the start of drawing of oneframe of display image and the completion of its output is equal to avalue that is obtained by adding the output time of one piece of outputunit data to the output cycle of one frame. That is, as compared with amode depicted in FIG. 11A, the required time is reduced by Δt, which isclose to the output cycle of one frame. This signifies that undistortedimages transmitted at time points t0, t1, t2, and so on can be displayedwith a small delay in a state suitable for the head-mounted display 100.As a result, the displayed images are unlikely to give a feeling ofstrangeness even if they are compared with a case where images initiallysuitable for the head-mounted display 100 are transmitted.

The foregoing description relates to processing inside the imagegeneration device 128. However, processing may be performed in a similarmanner even in a case where the image generation device 128 performsdecompression to decode source image data that is compression-encoded inthe content processing device 200, such as a cloud server, and issubjected to a streaming transfer. That is, the content processingdevice 200 and the image generation device 128 may perform compressionencoding, decompression decoding, and motion compensation on each ofunit areas obtained by dividing a frame plane.

Here, it is assumed that the unit areas are areas obtained byhorizontally dividing a predetermined number of rows of pixels, such asone row of pixels or two rows of pixels, or rectangular areas, forexample, of 16×16 pixels or 64×64 pixels, obtained by dividing thepixels in both horizontal and vertical directions. Each time processingtarget data on a unit area is acquired, the content processing device200 and the image generation device 128 start a compression-encodingprocess and a decompression-decoding process, respectively, and outputthe resulting processed data on an individual unit area basis. Thismakes it possible to further reduce the delay time for display,including the time required for data transmission from the contentprocessing device 200.

According to the present embodiment, which has been described above, anintegrated circuit for arranging a generated image in a format suitablefor display is mounted, for example, in a head-mounted display. Theintegrated circuit is configured so as to transmit transmission unitdata having a size smaller than one frame to a display panel wheneverthe transmission unit data is stored in a buffer memory. This makes itpossible to use, for example, a small-capacity SRAM as the buffermemory. As a result, the buffer memory can easily be mounted in the sameintegrated circuit.

In a case where a common method is employed for disposing a buffer forstoring the whole frame, for example, in a large-capacity DRAM, it isnecessary to transmit a large amount of data on drawn images. In thiscase, a transmission path is likely to become complex due to a problemwith a processing circuit board layout. For high-speed transmission, aplurality of DRAMs may be disposed in a parallel manner to reduce buswidth. However, the use of parallelly disposed DRAMs will increase thecost of production.

The small-capacity buffer memory according to the present embodimentmakes it possible to provide rapid access at a low cost and reduce thepower consumption for transmission. Further, the output to the displaypanel starts without waiting until one frame is drawn. This makes itpossible to minimize the time interval between the acquisition of anoriginal image and its display. In a case where a head-mounted displayis used, a problem is likely to occur with respect to the responsivenessof a display image to the motion of a user's head or to a useroperation. More specifically, the delay time for display may not onlydamage the sense of presence, but also cause physical conditiondeterioration such as visual motion sickness.

According to the present embodiment, conventional content not createdfor display on a head-mounted display can be displayed with a smalldelay. As a result, it is possible to easily and comfortably enjoyvarious videos through a head-mounted display without regard to adisplay format specified by content.

While the present disclosure has been described in conjunction with anembodiment, it will be understood by those skilled in the art that theembodiment is illustrative and not restrictive, and that the combinationof constituent elements and individual processes in the embodiment maybe variously modified, and further that such modifications also fallwithin the scope of the present disclosure.

What is claimed is:
 1. An image generation device that generates adisplay image to be viewed through an eyepiece disposed in front of adisplay panel, the image generation device comprising: a source imagereception control section that receives a source image from a contentprocessing device; a distorted image generation section that generatesdata on pixels in a display image obtained by giving the source image adistortion corresponding to the eyepiece; a partially distorted imagestorage section that stores the data on the pixels in an order of datageneration; and an image display control section that, whenever data ona predetermined number of pixels smaller than a total number of pixelsin the display image is stored in the partially distorted image storagesection, outputs the stored data to the display panel, wherein, thesource image reception control section constantly monitors reception ofeach row of pixels of the display image and instructs the distortedimage generation section to start a process of generating data for eachrow of pixels after it is determined that each row of pixels is receivedby the source image reception control section.
 2. The image generationdevice according to claim 1, wherein the distorted image generationsection generates the data on the pixels in the display image byreferencing a map, wherein the map is a displacement vector map thatindicates a positional relationship between pixels in the display imagegiven the distortion corresponding to the eyepiece and correspondingpixels in the source image.
 3. The image generation device according toclaim 2, wherein the map indicates the positional relationship atdiscrete positions having fewer pixels than the display image, andwherein, based on a positional relationship obtained by interpolatingthe positional relationship indicated by the map, the distorted imagegeneration section generates data on all pixels in the display image. 4.The image generation device according to claim 1, further comprising: adata transfer control section that, whenever data on the predeterminednumber of pixels is stored in the partially distorted image storagesection, exercises control so as to transmit the data.
 5. The imagegeneration device according to claim 1, further comprising: a userposture calculation section that acquires information regarding aposture of a head of a user wearing a head-mounted display having thedisplay panel; a user posture value storage section that stores theacquired information regarding the posture of the head of the user; anda view screen projection determination section that sets a view screenconfigured to define a plane of the display image in accordance with theposture of the head of the user, wherein the plane of the display imageis fixed with respect to a face of the user, and wherein the distortedimage generation section gives the distortion to an image projected onthe view screen.
 6. The image generation device according to claim 5,further comprising: a user controller instruction reception controlsection that acquires a user instruction determining whether the planeof the source image is to be fixed in a virtual space of a displaytarget or fixed to the display panel; and a user controller input valuestorage section that stores the user instruction, wherein, in a mode forfixing the plane of the source image in the virtual space of the displaytarget, the distorted image generation section gives the distortion tothe image projected on the view screen.
 7. The image generation deviceaccording to claim 1, wherein the partially distorted image storagesection includes a plurality of storage areas each having a capacity forstoring data on the predetermined number of pixels, and wherein thedistorted image generation section switches a storage location of thedata on the pixels in the display image between the plurality of storageareas.
 8. The image generation device according to claim 7, wherein aplurality of the storage areas included in the partially distorted imagestorage section each have a capacity that is an integer multiple of thecapacity of a unit area defined as a minimum unit of processing in thedistorted image generation section.
 9. The image generation deviceaccording to claim 1, wherein the source image is an undistorted imagegenerated for display on a flat-panel display.
 10. A head-mounteddisplay comprising: an image generation device that generates a displayimage to be viewed through an eyepiece disposed in front of a displaypanel, the image generation device including a source image receptioncontrol section that receives a source image from a content processingdevice, a distorted image generation section that generates data onpixels in a display image obtained by giving the source image adistortion corresponding to the eyepiece, a partially distorted imagestorage section that stores the data on the pixels in an order of datageneration, and an image display control section that, whenever data ona predetermined number of pixels smaller than a total number of pixelsin the display image is stored in the partially distorted image storagesection, outputs the stored data to the display panel; and the displaypanel that sequentially displays data outputted from the imagegeneration device, wherein, the source image reception control sectionconstantly monitors reception of each row of pixels of the display imageand instructs the distorted image generation section to start a processof generating data for each row of pixels after it is determined thateach row of pixels is received by the source image reception controlsection.
 11. A content processing system comprising: a head-mounteddisplay that includes an image generation device generating a displayimage to be viewed through an eyepiece disposed in front of a displaypanel, the image generation device including a source image receptioncontrol section that receives a source image, a distorted imagegeneration section that generates data on pixels in a display imageobtained by giving the source image a distortion corresponding to theeyepiece, a partially distorted image storage section that stores thedata on the pixels in an order of data generation, and an image displaycontrol section that, whenever data on a predetermined number of pixelssmaller than a total number of pixels in the display image is stored inthe partially distorted image storage section, outputs the stored datato the display panel; the display panel that sequentially displays dataoutputted from the image generation device; and a content processingdevice that generates the source image and transmits the generatedsource image to the head-mounted display, wherein, the source imagereception control section constantly monitors reception of each row ofpixels of the display image and instructs the distorted image generationsection to start a process of generating data for each row of pixelsafter it is determined that each row of pixels is received by the sourceimage reception control section.
 12. The content processing systemaccording to claim 11, wherein, when a source image transmitted from thecontent processing device is displayable as is, the image displaycontrol section outputs data on the source image to the display panel.13. An image display method used in an image generation device thatgenerates a display image that is to be viewed through an eyepiecedisposed in front of a display panel, the image display methodcomprising: receiving a source image from a content processing device;generating data on pixels in the display image that is obtained bygiving the source image a distortion corresponding to the eyepiece, andsequentially storing the generated data in a memory; constantly monitorreception of each row of pixels of the display image; begin a process ofgenerating data for each row of pixels after it is determined that eachrow of pixels is received; and whenever data on a predetermined numberof pixels smaller than a total number of pixels in the display image isstored in the memory, outputting the stored data to the display panel,constantly monitor reception of each row of pixels of the display image;begin a process of generating data for each row of pixels after it isdetermined that each row of pixels is received.
 14. A non-transitorycomputer readable medium having stored thereon a computer program for acomputer generating a display image that is to be viewed through aneyepiece disposed in front of a display panel, comprising: by a sourceimage reception control section, receiving a source image from a contentprocessing device; by a distorted image generation section, generatingdata on pixels in the display image that is obtained by giving thesource image a distortion corresponding to the eyepiece; by a partiallydistorted image storage section, sequentially storing the generated datain a memory; and by an image display control section, whenever data on apredetermined number of pixels smaller than a total number of pixels inthe display image is stored in the memory, outputting the stored data tothe display panel, wherein, the source image reception control sectionconstantly monitors reception of each row of pixels of the display imageand instructs the distorted image generation section to start a processof generating data for each row of pixels after it is determined thateach row of pixels is received by the source image reception controlsection.